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May  27,  1994 

MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION 

AND  TECHNOLOGY 

SUBJECT:  Audit  Report  on  Milestone  Review  Process  for  the  Advanced  Field 
Artillery  Tactical  Data  System  (Report  No.  94-115) 


We  are  providing  this  report  for  your  review  and  comments.  This  report  is  the 
second  in  a  series  of  reports  resulting  from  our  audit  of  the  milestone  review  process 
for  Component-managed  acquisition  programs.  We  requested  comments  on  a  draft  of 
this  report;  however,  comments  were  not  received. 

As  required  under  DoD  Directive  7650.3,  "Followup  on  General  Accounting 
Office,  DoD  Inspector  General,  and  Internal  Audit  Reports,"  September  5, 1989,  all 
audit  recommendations  must  be  resolved  promptly.  Therefore,  you  must  provide  final 
comments  on  the  recommendations  by  July  26,  1994.  The  recommendations  are 
subject  to  resolution  in  accordance  with  DioD  Directive  7650.3  in  the  event  of 
nonconcurrence  or  failure  to  comment. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  If  you  have  any 
questions  on  this  report,  please  contact  Mr.  Jack  D.  Snider,  Project  Manager,  at 
(703)  693-0402  (DSN  223-0402).  Appendix  G  lists  the  distribution  of  this  report. 
Audit  team  members  are  listed  inside  back  cover. 

David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 
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MILESTONE  REVIEW  PROCESS  FOR  THE  ADVANCED  FIELD 
ARTILLERY  TACTICAL  DATA  SYSTEM 


EXECUTIVE  SUMMARY 


Introduction.  The  Army's  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS) 
is  an  integrated  battlefield  management  and  decision  support  system.  The  system  is  one 
of  five  battlefield  automation  systems  of  the  Army  Tactical  Command  and  Control 
System.  The  AFATDS  utilizes  evolving  commercial  computer  technology  from  the 
Army  Tactical  Command  and  Control  System  hardware  and  software  procurement. 
The  AFATDS  Program  has  three  versions  of  software,  of  which  Version  1  is  scheduled 
for  a  Milestone  HI,  Production  Approval,  decision  in  December  1994  to  approve  entry 
into  the  production  and  deployment  phase  of  the  acquisition  process. 

Objective.  The  overall  audit  objective  was  to  evaluate  the  effectiveness  of  die 
milestone  review  process  for  Component-managed  acquisition  programs.  The 
AFATDS  was  one  program  selected  as  part  of  the  overall  audit.  We  are  reporting  our 
finding  and  recommendations  separately  because  management  needs  to  take  prompt 
corrective  action.  This  audit  assessed  the  adequacy  of  the  information  provided  to  DoD 
Component  milestone  decision  authorities  in  support  of  major  milestone  and  program 
reviews  and  evaluated  internal  controls  related  to  the  objective. 

Audit  Results.  The  AFATDS  Program  is  not  ready  to  proceed  into  the  production  and 
deployment  phase  of  the  acquisition  process.  The  AFATDS  software  to  be  deployed 
lacks  critical  capabilities  necessary  to  fulfill  user  requirements,  including 
communication  with  other  user  systems.  Subsequent  versions  of  AFATDS  software, 
potentially  capable  of  meeting  user  requirements,  do  not  have  a  dedicated  engineering 
and  manufacturing  development  phase  to  achieve  production  hardware  and  software 
configurations  suitable  for  deployment.  As  a  result,  the  Army  could  spend 
$187.2  million  for  hardware  that  does  not  meet  requirements,  spend  $4.6  million  for  an 
initial  operational  test  and  evaluation  that  will  not  prove  the  AFATDS  ready  for 
fielding,  experience  further  delays  in  the  development  of  software,  field  software  that 
does  not  meet  user  requirements,  and  support  two  systems  to  accomplish  the  same 
mission. 

Internal  Controls.  The  audit  did  not  identify  material  internal  control  weaknesses. 
Internal  controls  assessed  are  summarized  in  Part  I  of  this  report. 

Potential  Benefits  of  Audit.  Potential  monetary  benefits  are  $191.8  million  of 
procurement  and  operational  test  and  evaluation  funds  put  to  better  use,  of  which 
$76.9  million  will  occur  from  FYs  1994  through  1999.  Implementation  of  (he 
recommendations  will  provide  decisionmakers  all  available  information  to  make  fully 
informed  decisions  concerning  whether  the  AFATDS  Program  is  ready  to  proceed  into 
production  or  more  advanced  stages  of  development  and  whether  program  plans  for 
subsequent  stages  are  consistent  with  sound  acquisition  management  practices 
(Appendix  E). 


Summary  of  Recommendations.  We  recommended  designating  the  AFATDS 
Program  as  an  acquisition  category  ID  program,  canceling  the  Milestone  III  decision 
and  hardware  procurement  for  the  initial  version  of  software,  reporting  the  schedule 
baseline  breach,  requiring  a  Defense  Acquisition  Board  program  review  of  alternatives 
for  meeting  user  requirements,  updating  the  AFATDS  Operational  Requirements 
Document,  requiring  revision  of  the  Test  and  Evaluation  Master  plan  to  reflect  the 
minimum  operating  requirements,  and  requiring  that  a  cost  and  operational 
effectiveness  analysis  is  completed  before  the  recommended  Defense  Acquisition  Board 
program  review. 

Management  Comments.  We  requested  comments  on  a  draft  of  this  report  from  the 
Under  Secretary  of  Defense  for  Acquisition  and  Technology;  however,  comments  were 
not  received.  As  a  result,  we  request  comments  on  this  final  report  by  July  26,  1994. 
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The  Acquisition  Management  Directorate,  Office  of  the  Assistant  Inspector  General  for 
Auditing,  DoD,  prepared  this  report. 


Part  I  -  Introduction 
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Introduction 


Background 


This  report  discusses  the  adequacy  of  the  information  provided  to  the  Army 
milestone  decision  authority  in  support  of  a  Milestone  HI,  Production  Approval, 
decision*  for  the  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS). 


Decision  Authority.  The  DoD  Instruction  5000.2,  "Defense  Acquisition 
Policies  and  Procedures,"  February  23,  1991,  states  that  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology  shall  be  the  acquisition  program 
milestone  decision2  authority  for  all  acquisition  category  (ACAT)  P  programs. 
However,  the  Under  Secretary  can  delegate  decision  authority  to  the  cognizant 
DoD  Component  head,  who  may  delegate  authority  to  the  DoD  Component 
acquisition  executive.  ACAT  I  programs  administered  by  the  Under  Secretary 
and  the  DoD  Component  head  are  titled  ACAT  ID  and  ACAT  IC  programs, 
respectively. 


Advanced  Field  Artillery  Tactical  Data  System.  The  Army’s  AFATDS,  an 
ACATIC  Program,  is  to  provide  an  integrated  battlefield  management  and 
decision  support  system,  designed  to  overcome  the  size,  vulnerability,  high 
sustainment  cost,  limited  functionality,  central  processing,  and  training 
limitations  of  the  Tactical  Fire  Direction  System  (TACFIRE).  The  AFATDS 
Program  is  one  of  five  battlefield  automation  systems  of  the  Army  Tactical 
Command  and  Control  System.  The  AFATDS  will  automate  27  fire  support 
functions,  grouped  in  five  fire  support  operational  requirements:  fire  support 
execution,  fire  support  planning,  movement  control,  field  artillery  mission 
support,  and  field  artillery  fire  direction  operations.  The  AFATDS  will  utilize 
the  evolving  commercial  computer  technology  selected  for  the  Army  Tactical 
Command  and  Control  System  architecture.  The  AFATDS  Program  is 
comprised  of  three  versions: 


o  Version  1  was  intended  to  be  interoperable  with  TACFIRE  and  to 
provide  division  and  corps-level  initial  functionality  to  include  attack  by  cannon, 
missile,  or  rocket;  naval  gunfire;  mortars  and  air  attack  systems  employment. 


1  Appendix  A  discusses  the  specifics  of  a  Milestone  HI,  Production  Approval,  decision. 

2The  point  when  a  recommendation  is  made  and  approval  sought  regarding  starting  or 
continuing  (proceeding  to  next  phase)  an  acquisition  program.  Milestones  are  0  (Concept 
Studies  Approval),  I  (Concept  Demonstration),  II  (Development  Approval),  HI  (Production 
Approval),  and  IV  (Major  Modification  Approval). 

3 An  ACAT  I  designation  is  issued  to  all  major  Defense  acquisition  programs  that  have  an 
eventual  total  expenditure  for  research,  development,  acquisition,  and  evaluation  of  more  than 
$300  million  in  FY  1990  constant  dollars  or  an  eventual  total  expenditure  for  procurement  of 
more  than  $1.8  billion  in  FY  1990  constant  dollars. 

^TACFIRE  has  been  provided  to  all  heavy  divisions  and  corps  of  the  active  force.  TACFIRE 
consists  of  two  types  of  central  computers,  providing  field  artillery  fire  planning  and  tactical  fire 
control,  and  a  remote  terminal,  providing  communications  with  the  central  computers.  Tactical 
fire  control  includes  evaluating  targets,  selecting  units  to  fire,  munitions,  and  volume  of  fire. 
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o  Version  2  will  replace  TACFIRE  in  FY  1996  and  will  be  fielded  in 
two  increments,  increasing  automation  to  provide  enhanced  capabilities  in  target 
generation  and  prioritization,  attack  system  analysis,  naval  gunfire,  and  close  air 
support.  It  will  also  provide  improved  fire  support  planning  and  mcreased 
interoperability.  To  satisfy  higher  echelon  operational  considerations,  system 
software  will  have  greater  flexibility  for  continuity  of  operations  (CONOPS) 
alternatives. 

o  Version  3  is  the  objective  system  and  will  fully  automate  the 
AFATDS  system.  It  will  provide  computerized  maps,  integrate  chemical  and 
conventional  schedule  of  fires,  provide  advanced  guidance  development, 
provide  technical  fire  direction,  and  replace  the  Fire  Direction  Data  Manager 
system.5 

The  December  31,  1992,  AFATDS  Selected  Acquisition  Report  estimated  a 
total  acquisition  cost  of  $876.8  million  in  then-year  dollars  of  which 
$394.2  million  has  been  appropriated  through  FY  1994: 

o  $300.8  million  for  research,  development,  test,  and  evaluation  and 

o  $93.4  million  for  procurement. 

The  FY  1994  procurement  appropriation  of  $12  million  was  $12  million  less 
than  the  budget  request  of  $24  million. 

Under  cost-plus-award-fee  contract  DAAB-90-C-E708,  valued  at 
$69.9  million,  Magnavox  Electronic  Systems  Company,  Fort  Wayne,  Indiana, 
is  developing  AFATDS  Version  1  software.  Under  an  option  to  the  contract, 
Version  2  software  is  being  developed  for  $43  million.  Die  AFATDS 
Version  1  software  is  scheduled  for  a  Milestone  IE,  Production  Approval, 
decision  in  December  1994. 


Objective 


The  overall  audit  objective  was  to  evaluate  the  effectiveness  of  the  milestone 
review  process  for  Component-managed  acquisition  programs.  The  audit  also 
assessed  the  adequacy  of  the  information  provided  to  DoD  Component 
milestone  decision  authorities  in  support  of  major  milestone  and  program 
reviews  and  evaluated  internal  controls  related  to  the  objective.  The  AFATDS 
was  one  program  reviewed  during  the  audit.  We  determined  that  management 
attention  was  needed  on  the  AFATDS  Program  as  it  proceeded  toward  a 


5The  Fire  Direction  Data  Manager  system  (the  system)  improves  the  Multiple  Launch  Rocket 
System  Fire  Direction  System  by  increasing  the  fire  direction  system  processing,  storage,  and 
communications  capability.  The  system  provides  tactical  fire  control  of  rockets  not  possible 
with  TACFIRE.  The  system  will  be  deployed  in  FY  1994. 
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Milestone  m,  Production  Approval,  decision  and  before  the  completion  of  our 
overall  audit  work.  Therefore,  we  are  reporting  this  issue  separately  before  the 
conclusion  of  our  overall  audit. 


Scope  and  Methodology 


We  conducted  this  program  audit  from  May  1993  through  Januaiy  1994  and 
reviewed  data  dated  from  September  1986  through  January  1994.  To 
accomplish  the  objective,  we: 

o  discussed  issues  relating  to  the  effectiveness  of  the  milestone  review 
process  for  the  AFATDS  Program  with  Office  of  the  Secretary  of  Defense  and 
Army  personnel; 

o  determined  the  adequacy  of  the  information  that  the  Army  provided  to 
the  decision  authorities  in  support  of  major  milestone  and  program  reviews; 

o  evaluated  the  effectiveness  of  the  milestone  review  and  program 
review  processes  for  the  AFATDS  Program; 

o  reviewed  AFATDS  Program  decision  documents  as  well  as  selected 
acquisition  reports,  Defense  acquisition  executive  summary  reports,  and  various 
contract  cost  management  reports;  and 

o  examined  contract  DAAB-90-C-E708,  valued  at  $69.9  million,  with 
Magnavox  Electronic  Systems  Company,  Fort  Wayne,  Indiana. 

The  audit  was  made  in  accordance  with  auditing  standards  issued  by  the 
Comptroller  General  of  the  United  States,  as  implemented  by  the  Inspector 
General,  DoD,  and  accordingly  included  such  tests  of  internal  controls  as  were 
deemed  necessary.  We  did  not  rely  on  computer-processed  data  to  support  the 
finding  and  recommendations  in  this  audit  report.  Appendix  F  lists  the 
organizations  visited  or  contacted. 


Internal  Controls 


Internal  Controls  Evaluated.  We  evaluated  internal  controls  related  to  the 
effectiveness  of  the  milestone  review  process  and  the  adequacy  of  the 
information  provided  to  the  milestone  decision  authorities  in  support  of  major 
milestone  and  program  reviews  for  the  AFATDS  Program.  The  DoD 
Instruction  5000.2,  "Defense  Acquisition  Policies  and  Procedures, 
February  23,  1991,  and  DoD  Manual  5000.2-M,  "Defense  Acquisition 
Management  Documentation  and  Reports,"  February  23,  1991,  specify  those 
controls  and  procedures.  We  also  assessed  implementation  of  the  requirements 
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of  DoD  Directive  5010.38,  "Internal  Management  Control  Program,"  April  14, 
1987,  including  performance  of  vulnerability  assessments  and  management 
control  reviews. 

Interna!  Control  Weakness  Not  Identified.  The  audit  did  not  identify  any 
material  internal  control  weakness,  as  defined  by  DoD  Directive  5010.38. 
Existing  internal  controls,  if  properly  implemented,  were  sufficient  to  preclude 
the  deficiencies  noted  in  this  report.  In  the  AFATDS  FY  1993  Statement  of 
Internal  Management  Control,  August  20,  1993,  no  internal  control  deficiencies 
were  identified.  As  a  part  of  this  audit,  we  did  not  examine  the  effectiveness  of 
implementation  of  the  DoD  Internal  Management  Control  Program  for  DoD 
Component-managed  programs  because  our  objectives  were  limited  to  AFATDS 
program  management.  Our  summary  report  on  the  overall  audit  will  include 
our  assessment  of  the  internal  controls  for  Component-managed  programs. 


Prior  Audits  and  Other  Reviews 


Since  1988,  the  Office  of  the  Inspector  General,  DoD,  issued  one  report  that 
included  the  AFATDS  Program.  However,  we  did  not  follow  up  on  the  prior 
audit  report  because  it  did  not  contain  findings  or  recommendations  related  to 
our  objective. 


Other  Matters  of  Interest 


On  March  11,  1994,  the  Army  issued  Magnavox  Electronic  Systems  Company  a 
stop  work  order.  The  order  stated  that  the  Government  was  not  satisfied  with 
the  contractor's  progress  in  completion  of  AFATDS  Version  1  software.  The 
order  required  that  all  work  on  Version  2  software  stop  on  or  before  April  1, 
1994,  and  expected  the  contractor  to  deliver  Version  1  in  accordance  with  the 
requirements  of  the  contract.  The  order  also  stated  that  because  of  the  software 
development  delays  and  an  anticipated  contractor  inability  to  deliver  an 
acceptable  product  in  time  for  a  July  1994  initial  operational  test  and  evaluation 
(IOT&E)  (Appendix  D),  the  Army  reduced  the  operational  testing  for  FY  1994. 
A  separate  AFATDS  IOT&E  to  validate  fire  support  functionality  will  be 
scheduled  during  FY  1995.  In  preparation  for  that  test,  complete  Version  1 
software  must  be  delivered  to  the  Army  for  technical  testing  by  June  20,  1994. 
Based  upon  the  contractor's  performance  on  Version  1,  the  Army  will  make  a 
final  decision  on  Version  2. 
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Part  II  -  Finding  and  Recommendations 
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Proceeding  Into  Production  and 
Deployment 

The  AFATDS  Program  is  not  ready  to  proceed  into  production  and 
deployment.  The  Version  1  of  AFATDS  software  scheduled  for  a 
Milestone  HI,  Production  Approval,  decision  in  December  1994  lacks 
critical  capabilities  necessary  to  fulfill  user  requirements,  including 
communication  with  other  user  systems.  Subsequent  versions  of 
AFATDS  software,  potentially  capable  of  meeting  user  requirements,  do 
not  have  a  dedicated  phase  of  engineering  and  manufacturing 
development  needed  to  achieve  production  hardware  and  software 
configurations  suitable  for  deployment.  The  lack  of  readiness  to  enter 
production  and  deployment  was  caused  by  the  Army  decision  to  deploy 
Version  1  of  AFATDS  instead  of  Version  2  after  schedule  slippage 
resulting  from  Version  1  software  development  problems.  As  a  result, 
the  Army  could: 

o  spend  $187.2  million  for  hardware  that  does  not  meet 
requirements, 

o  spend  $4.6  million  for  an  initial  operational  test  and  evaluation 
that  will  not  prove  whether  the  AFATDS  is  ready  for  fielding, 

o  experience  further  delays  in  the  development  of  Version  2 
software,  and 

o  support  two  systems  to  accomplish  the  same  mission. 


Background 

The  DoD  Instruction  5000.2  states  that  computer  resources  are  hardware, 
firmware,6  software,  documentation  services,  support  services,  supplies,  and 
spare  parts.  (Appendix  B  further  discusses  computer  resources.)  The  Defense 
Systems  Management  College's  Mission-Critical  Computer  Resources 
Management  Guide  further  states  that: 

o  computer  resources  are  integral  to  weapon  systems  because  software  is 
critical  for  weapon  system  development; 

o  software  development  costs  can  exceed  initial  budget  estimates  by 
50  percent  to  100  percent; 

o  the  performance  of  modem  weapon  systems  is  largely  dependent  on 
the  quality  of  computer  resources;  and 


^Software  that  is  stored  on  a  fixed  system,  such  as  the  read-only  memory  of  the  system. 
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o  when  software  falls  behind  schedule,  development  lead  times  cannot 
be  shortened  by  applying  more  resources.  Therefore,  only  more  time  can  help 
fix  schedule  problems. 


Assessing  Acquisition  Strategy 


The  AFATDS  Version  1  software  lacks  critical  capabilities  necessary  to  fulfill 
user  requirements,  including  communication  with  other  user  hardware.  The 
AFATDS  Version  2  software,  potentially  capable  of  meeting  user  requirements, 
does  not  have  a  dedicated  phase  of  engineering  and  manufacturing  development 
needed  to  achieve  production  hardware  and  software  configurations  suitable  for 
deployment.  Until  AFATDS  can  fulfill  user  requirements,  the  Army  and  the 
Marine  Corps  are  fielding  an  alternative  system. 


Capabilities.  The  AFATDS  Program  Office  hopes  to  gain  permission,  through 
the  Milestone  m,  Production  Approval,  decision,  to  buy  hardware  and 
designate  the  Program  Executive  Officer,  Command  and  Control  Systems, 
(Appendix  C)  as  the  decision  authority  on  future  versions  of  the  software. 
Therefore,  a  favorable  milestone  decision  will  remove  the  AFATDS  Program 
from  the  oversight  of  the  Assistant  Secretary  of  the  Army  (Research, 
Development  and  Acquisition)  even  though  the  AFATDS  Program  will  not  have 
demonstrated  that  it  will  satisfy  requirements. 


Acquisition  Documentation.  The  acquisition  documentation  did  not 
fairly  state  the  position  of  the  AFATDS  Program.  The  Version  1  software  will 
not  deliver  the  command  and  control  functionally  needed  to  satisfy  user 
requirements.  Specifically, 

o  The  Version  1  software  does  not  provide  sufficient  artillery 
target  intelligence,  close  air  support,  and  naval  gunfire  functionality. 

o  The  AFATDS  Version  1  software  is  run  on  a  tactical  computer 
unit  (TCU)  and  a  lightweight  computer  unit  (LCU);  however,  the  Version  1 
software  will  not  allow  the  TCU  and  LCU  to  share  the  same  local  area  network. 


o  The  acquisition  strategy  does  not  permit  the  user  to  provide 
feedback  for  the  Version  2  software  development. 


o  The  test  and  evaluation  strategy  is  inadequate, 
o  The  computer  hardware  may  not  possess  sufficient  speed  and 

memory. 

Artillery  Target  Intelligence.  The  AFATDS  Operational  Requirements 
Document  (ORD)  does  not  specify  whether  or  when  artillery  target  intelligence' 

^  Artillery  target  intelligence  is  the  ability  of  the  software  to  locate  a  target  from  various  sensor 
readings. 
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should  be  a  part  of  the  fielded  AFATDS  software;  however,  the  Unites  States 
Army  Field  Artillery  School  (USAFAS)  (Appendix  C)  states  artillery  target 
intelligence  would  improve  AFATDS  fire  support  functionality.  The  DoD 
Manual  5000.2-M  states  that  the  ORD  shall  clearly  specify  the  operational 
capability  or  level  of  performance  necessary  to  declare  initial  and  full 
operational  capability.  The  AFATDS  Version  1  software  will  not  have  artillery 
target  intelligence  capability;  however,  the  AFATDS  Program  Office 
documented  that  it  will  be  a  feature  of  either  Version  2  or  2.1.  Artillery  target 
intelligence  offers  needed  functionality  that  is  currently  available  from  other 
automated  fire  support  programs.  Accordingly,  to  enhance  fire  support 
capability,  AFATDS  must  offer  improvements  over  fielded  systems.  Therefore, 
the  ORD  should  be  updated  to  reflect  the  need  for  artillery  target  intelligence. 

Close  Air  Support  and  Naval  Gunfire.  Version  1  of  the  AFATDS 
software  will  not  be  able  to  effectively  utilize  all  fire  support  assets  available  to 
the  commander  as  the  ORD  requires.  Specifically,  the  Marine  Corps  stated  that 
the  Version  1  software  will  not  be  able  to  perform  sufficient  close  air  support 
and  coordinate  naval  gunfire.  The  AFATDS  ORD  requires  that  close  air 
support  and  naval  gunfire  functionality  be  available  on  the  fielded  software; 
however,  the  AFATDS  Program  Office  stated  that  the  Version  2  software  will 
specifically  address  this  requirement.  The  Marine  Corps  stated  that  adding 
those  functions  to  the  Version  2  software  will  allow  it  to  evaluate  the  system  for 
fielding.  Additionally,  Version  1  does  not  offer  sufficient  functionality  to 
employ  Air  Force  aircraft  in  close  air  support  roles.  Therefore,  the  design  of 
the  Version  1  software  will  not  satisfy  the  requirements  of  managing  all 
available  fixe  support  assets  through  a  single  system.  Naval  gunfire  and  close 
air  support  functionality  are  necessary  to  provide  significant  additional 
capabilities  over  TACFIRE  and  the  Initial  Fire  Support  Automation  System 
(IFSAS)  Program;  therefore,  it  would  be  inappropriate  to  field  the  AFATDS 
software  until  the  AFATDS  Program  has  undergone  sufficient  engineering  and 
manufacturing  development  to  ensure  that  this  functionality  is  available. 

Memory.  The  AFATDS  hardware  may  not  possess  sufficient  memory 
(Appendix  B)  capacity  to  satisfy  mission  requirements.  The  DoD  Instruction 
5000.2  states  that  a  program  office  will  not  finalize  computer  hardware  resource 
decisions  until  the  software  design  is  mature  enough  to  minimize  the  risk  that 
the  computer  selected  has  inadequate  speed  and  memory  capadfy;  however,  it 
does  not  provide  a  quantitative  means  to  measure  this  requirement.  The 
Mission-Critical  Computer  Resources  Management  Guide  does  quantify  this 
requirement  by  stating  a  computer  should  not  use  more  that  50  percent  of  speed 
and  memory  to  accommodate  a  system's  growth  over  its  life  cycle.  Speed  and 
memory  are  interrelated  in  that  insufficient  random  access  memory”  can  lead  to 
the  need  for  frequent  exchanges  between  internal  and  secondary  memory,  which 
reduces  the  effective  speed  of  the  computer.  The  Project  Office  states  that  the 
LCU  must  have  a  minimum  of  32  megabytes  of  random  access  memory; 


^Random  access  memory  is  semiconductor-based  memory  that  die  computer  can  read  and  write 
to. 


10 


Proceeding  Into  Production  and  Deployment 


however,  the  Reduced  Instruction  Set  Computer  (RISC)9  TCU  must  have 
considerably  more  than  32  megabytes  of  random  access  memory  due  to  its 
design.  Currently,  the  TCU  and  LCU  have  32  megabytes  of  random  access 
memory. 


Memory  Capacities.  The  AFATDS  Program  Office  estimated 
that  current  random  access  memory  capacities  can  probably  support  the 
additional  software  without  further  increase  in  memory  if  performance 
requirements  are  not  increased  and  large  Common  Army  Tactical  Command  and 
Control  System  Support  Software10  applications  are  not  added.  For  example,  if 
the  Terrain  Evaluation  Module  Software  is  added,  current  processing  capacity 
will  not  be  sufficient  to  perform  intensive  calculations  concurrently  on  the  same 
computer  that  is  performing  AFATDS  fire  mission-type  processing.  The  Army 
stated  that  the  speed  and  complexity  of  tomorrow's  battlefields  will  continue  to 
increase  and  that  a  higher  degree  of  automation  is  required  to  plan,  direct,  and 
execute  combat  operations  effectively.  Versions  2  and  3  of  the  AFATDS 
software  are  expected  to  increase  by  25  and  10  percent,  respectively;  however, 
the  Version  1  computer  will  not  be  able  to  accommodate  those  increases  without 
compromising  functionality.  Specifically,  if  the  user  identifies  additional 
functionality  necessary  to  enhance  mission  effectiveness,  the  current  capacity 
will  not  allow  the  change  without  eliminating  existing  functionality  or  degrading 
system  performance.  These  conditions  exist  because  the  AFATDS  ORD  does 
not  specify  what  minimum  speed  and  memory  reserves  are  necessary  to  field  the 
software.  Additionally,  the  acquisition  strategy  does  not  provide  sufficient  time 
for  the  user  to  evaluate  the  completed  software  on  the  target  computers  before 
the  Milestone  HI,  Production  Approval,  decision.  The  Marine  Corps  stated  that 
more  developmental  tests  of  the  AFATDS  Program  with  actual  user  operational 
units  is  necessary  to  determine  whether  AFATDS  meets  the  users'  requirements. 

Hardware.  From  1994  through  2004,  the  AFATDS  Program 
plans  to  purchase  2,272  TCU  and  380  LCU  computers  for  approximately 
$179  million  and  $8.2  million  in  then-year  dollars,  respectively,  with  an 
estimated  10-year  useful  life.  The  Army  plans  to  buy  1,803  additional  LCU 
computers  identical  to  the  AFATDS  LCUs  for  the  EFSAS  and  Fire  Support  Ada 
Conversion  Programs  because,  once  it  is  fielded,  AFATDS  is  intended  to 
replace  the  IFSAS  and  Fire  Support  Ada  Conversion  Programs.  At 
32  megabytes  of  random  access  memory,  neither  the  RISC  TCU  nor  the  LCU 
will  have  a  random  access  memory  reserve. 


9 xhe  RISC  is  a  microprocessor  design  based  on  the  premise  that  most  instructions  that  a 
computer  decodes  and  executes  are  simple;  therefore,  RISC,  chips  limit  the  number  of 
instructions  built  into  a  microprocessor  but  optimizes  each  for  rapid  execution.  The  RISC  chips 
are  slower  than  general  purpose  complex  instruction  set  chips  when  complex  instructions  must 
be  executed. 

l°Common  Army  Tactical  Command  and  Control  System  Support  Software  incorporate  generic 
command  and  control  support  functions  found  in  all  Army  Tactical  Command  and  Control 
System  Battlefield  Functional  Area  Control  Systems.  The  support  software  consists  of  common 
system  support  software  and  application  software,  including  Movement  Control  Module,  Digital 
Mapping  Module,  and  Terrain  Evaluation  Module. 


11 


Proceeding  Into  Production  and  Deployment 


Additional  Memory.  The  AFATDS  Program  Office  is 
considering  adding  an  additional  48  megabytes  of  random  access  memory  to  the 
TCU,  upgrading  the  LCU  from  25  to  66  megahertz  for  additional  speed,  or 
adding  a  RISC-based  LCU  with  128  megabytes  of  memory.  However,  a  formal 
plan  has  not  been  established,  life-cycle  cost  estimates  do  not  reflect  these 
changes,  and  the  proposed  hardware  has  not  been  tested  to  determine  whether  it 
will  satisfy  a  50-percent  margin  requirement.  Since  the  AFATDS  Program 
must  have  sufficient  capacity  to  meet  current  and  future  requirements,  the 
AFATDS  Program  Office  must  verify  that  the  hardware  does  not  hamper 
system  growth.  Specifically,  procurement  of  TCU  and  LCU  computers  should 
be  postponed  until,  at  a  minimum,  Version  2  software  is  available  and  fully 
tested  on  representative  hardware  platforms.  Further,  the  decision  must  first  be 
made  as  to  whether  the  AFATDS  software  should  even  be  fielded  and,  if  so, 
what  version  should  be  fielded,  based  on  the  results  of  the  ongoing  cost  and 
operational  effectiveness  analysis  (COEA)  scheduled  for  completion  in 
November  1994.  Thereafter,  the  acquisition  strategy,  life-cycle  cost  estimates, 
the  test  and  evaluation  master  plan  (TEMP),  and  the  ORD  should  be  updated  to 
reflect  those  requirements  before  proceeding  with  procurement  actions. 

AFATDS  Architecture.  The  design  of  the  AFATDS  Version  1 
software  architecture  wall  not  satisfy  mission  requirements.  The  DoD 
Instruction  5000.2  states  a  program  may  not  enter  full-rate  production  unless  the 
milestone  decision  authority  confirms  that  performance  objectives  and 
thresholds  have  been  validated  and  test  results  and  low-rate  initial  production 
reasonably  assure  that  the  design  is  stable,  operationally  acceptable,  and 
logistically  supportable.  The  AFATDS  Project  Office  is  developing  versions  of 
the  software  with  a  heterogeneous  architecture,  1  however  Version  1  is  designed 
on  a  homogeneous  architecture12  that  allows  the  system  to  send  messages  at  the 
bit  level,  ones  and  zeros,  rather  than  messages,  for  speed  in  execution.  The 
Army  Materiel  Systems  Analysis  Activity  (AMS  A  A)  (Appendix  C)  stated  that 
AFATDS  software  was  originally  intended  to  run  only  on  the  TCU  computer. 
However,  as  the  AFATDS  Program  progressed,  the  AFATDS  Program  Office 
added  the  LCU  through  an  engineering  change  proposal.  The  user  stated  that 
those  computers  have  different  processors  and  will  not  be  capable  of  sharing  the 
same  data  via  the  local  area  network  or  performing  CONOPS  procedures  for 
each  other;  therefore,  information  must  be  shared  by  radio  or  other 
communication  devices.  The  heterogeneous  architecture  will  allow  TCU  and 
LCU  operational  facilities  to  share  automated  information  and  provide 
CONOPS  support  for  each  other.  The  lack  of  a  heterogeneous  architecture  for 
Version  1  software  is  a  significant  operational  shortfall;  however,  the  AFATDS 
Project  Office  has  not  planned  to  address  this  problem.  The  AFATDS 


^Heterogeneous  architecture  allows  communication  between  computers  by  sending  messages 
with  the  information.  The  messages  tell  the  other  computer  how  to  read  the  information; 
therefore,  this  process  allows  computers  with  different  processors  to  be  part  of  the  same  local 
area  network. 

^Homogeneous  architecture  requires  all  computers  within  the  local  area  network  to  have 
identical  computer  processors  from  the  same  manufacturer;  however,  the  AFATDS  will  have 
two  processors,  the  TCU  and  LCU. 
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architecture  should  reflect  known  requirements,  especially  when  seeking  a 
production  approval  decision.  Also,  the  ORD  should  state  those  requirements 
explicitly  before  a  decision  is  made  to  field  software. 

Alternative  System.  The  Army  has  an  urgent  need  for  the  replacement  of 
TACFIRE;  however,  AFATDS  acquisition  documentation  could  lead 
decisionmakers  to  allow  premature  fielding  of  AFATDS  Version  1  software. 
The  DoD  Instruction  5000.2  states  that  documentation  is  the  primary  means  for 
the  functional  staff  and  the  program  manager  to  provide  the  milestone  decision 
authority  with  the  information  needed  for  a  milestone  decision.  In  addition,  in  a 
November  18,  1991,  memorandum  to  the  Army  Acquisition  Executive,  the 
Under  Secretary  of  Defense  for  Acquisition  required  that  OSD  staffs  continue  to 
monitor  the  Army  Tactical  Command  and  Control  System  Programs  through 
Defense  acquisition  executive  summary  reports  to  maintain  visibility  in  the 
Army  Tactical  Command  and  Control  System  Programs.  In  its  mission 
description  section,  the  AFATDS  Defense  Acquisition  Executive  Summary 
report  stated  that  AFATDS  software  is  to  replace  TACFIRE  and  shows  that 
Version  1  will  be  fielded  to  the  total  force.  However,  the  report  did  not  inform 
decisionmakers  that  the  Army  and  Marine  Corps  have  fielded  the  IFSAS 
Program.  Additionally,  the  AFATDS  ORD  and  TEMP  do  not  adequately 
disclose  that  the  Army  has  fielded  IFSAS  to  overcome  TACFIRE  limitations 
pending  the  availability  of  AFATDS. 

initial  Fire  Support  Automation  System  Capabilities.  The  IFSAS 
Program  consists  of  a  C  software  language  version  of  TACFIRE  that  can  run  on 
the  AFATDS  LCU,  while  only  utilizing  3  megabytes  of  the  32-megabyte 
random  access  memory.  The  IFSAS  Program  has  more  functionality  than 
TACFIRE  and  was  developed  by  the  Field  Artillery  Tactical  Data  System 
Project  Office  (Appendix  C)  at  the  request  of  the  Commandant,  USAFAS,  to 
provide  a  means  to  replace  Battalion  TACFIRE  quickly  and  economically.  The 
IFSAS  Program  will  be  capable  of  performing  artillery  target  intelligence; 
however,  it  does  not  perform  close  air  support  and  naval  gunfire  functions 
adequately.  The  IFSAS  Program  is  not  intended  to  provide  the  level  of 
automation  that  would  be  expected  from  the  objective  AFATDS  system,  but  it 
has  been  designed  to  provide  immediate  relief  for  TACFIRE  and  can 
accommodate  additional  functionality.  Therefore,  the  Army  has  achieved 
through  the  IFSAS  Program  a  level  of  automation  that  will  relieve  pressure 
temporarily  to  field  AFATDS  software. 

Cost  and  Operational  Effectiveness  Analysis.  According  to  Army 
Training  and  Doctrine  Command  officials,  the  IFSAS  Program  will  remain  in 
the  Army  inventory  whether  or  not  AFATDS  is  fielded.  The  Training  and 
Doctrine  Command  is  preparing  the  AFATDS  COEA  using  three  alternatives 
planned  to  satisfy  objective  system  requirements  in  the  year  2004.  The 
three  alternatives,  which  all  include  IFSAS,  are: 

o  stop  AFATDS  development  and  field  IFSAS  to  the  entire 

Army, 


o  stop  further  AFATDS  development  and  field  Version  1  to 
selected  units  with  IFSAS  to  the  remainder  of  the  Army,  or 
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o  continue  AFATDS  development  and  field  Version  3  to  selected 
nnits  with  IFSAS  to  the  remainder  of  the  Army. 

Initial  Fire  Support  Automation  System  Fielding.  The  Marine  Corps 
is  utilizing  portions  of  the  TACFIRE  program  but  intends  to  field  the  IFSAS 
Program  in  the  fourth  quarter  of  FY  1994  or  until  the  AFATDS  Version  2 
software  demonstrates  that  it  will  satisfy  requirements.  The  Army  and  Marine 
Corps  deployment  of  IFSAS  Program  will  directly  impact  AFATDS 
requirements  as  well  as  AFATDS  test  and  evaluation;  however,  the  AFATDS 
ORD  and  TEMP  do  not  address  the  effects  of  the  combined  fielding  of  IFSAS 
and  AFATDS.  To  adequately  assess  the  AFATDS  software  for  fielding,  the 
AFATDS  test  and  evaluation  program  must  consider  the  increased  fire  support 
capabilities  offered  by  IFSAS  rather  than  TACFIRE  performance  alone. 
Therefore,  the  recent  fielding  of  IFSAS  indicates  that  the  fielding  of  AFATDS 
software  must  noticeably  improve  fire  support  functionality,  which  will  require 
additional  engineering  and  manufacturing  development.  Also,  decisionmakers 
must  have  timely  access  to  all  available  information  on  the  IFSAS  Program 
through  the  ORD,  TEMP,  and  COEA  to  make  fully  informed  decisions  on 
automated  fire  support  systems.  The  AFATDS  Version  1  software  should  not 
be  fielded  and  a  dedicated  phase  of  engineering  and  manufacturing  development 
would  be  required  to  provide  a  version  of  AFATDS  software  that  satisfies  user 
requirements.  Additionally,  the  ORD  and  the  TEMP  should  be  revised  to 
reflect  the  impact  of  IFSAS  on  fielding  AFATDS. 


Cause  for  Program  Position 


The  AFATDS  Project  Office  has  based  acquisition  decisions  on  schedule 
considerations  at  the  expense  of  functionality.  The  DoD  Directive  5000.1, 
"Defense  Acquisition,"  February  23,  1991,  states  that  a  program  shall  follow  an 
event-based  acquisition  strategy;  however,  management  decisions  concerning 
development,  test,  and  evaluation  and  IOT&E  were  based  on  schedule  concerns. 
Additionally,  the  AFATDS  documentation  does  not  provide  the  decisionmaker 
with  the  necessary  information  to  make  fully  informed  decisions. 

Acquisition  Oversight.  The  OSD  recognizes  that  the  AFATDS  Program  is 
critical  to  future  fire  support  operations  and  has  attempted  to  provide  guidance 
to  the  AFATDS  Program  by  monitoring  the  AFATDS  Program's  Defense 
Acquisition  Executive  Summary  reports;  however,  OSD  oversight  has  not  been 
sufficient  to  prevent  the  AFATDS  Program  from  following  a  schedule-driven 
acquisition  strategy.  By  monitoring  only  the  AFATDS  Program's  Defense 
Acquisition  Executive  Summary  reports,  OSD  did  not  obtain  sufficient 
information  to  determine  that  the  Version  1  software  design  will  not  meet  user 
requirements,  the  Force  Development  Testing  and  Experimentation  (FDT&E) 
will  not  be  accomplished  with  the  target  computers  or  the  complete  software, 
and  the  IOT&E  has  a  high  risk  of  failure.  Additionally,  monitoring  the  Defense 
Acquisition  Executive  Summary  reports  alone  does  not  disclose  that  after  a 
Milestone  HI  decision  on  the  Version  1  software  all  future  fielding  decisions 
would  be  made  by  the  Program  Executive  Officer,  Command  and  Control 
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Systems.  The  OSD  should  revise  the  AFATDS  Program  acquisition  strategy  to 
include  elevating  the  Program  to  ACAT  ID  status  to  verify  that  the  AFATDS 
Program  satisfies  user  requirements.  The  new  acquisition  strategy  would  base 
decisions  on  events  and  the  elevated  acquisition  category  would  verify  that 
sufficient  information  was  made  available  to  OSD  officials  to  support  the 
decisions. 

Assessments.  The  Version  1  software  is  to  provide  functionality  greater  than 
TACFIRE.  However,  the  Army  Fire  Support  Command,  Control,  and 
Communications  Automation  Plan,  December  1992,  states  Version  1  will  only 
have  functionality  equivalent  to  TACFIRE  and  that  Version  2  will  replace 
TACFIRE  and  possess  superior  functionality.  The  USAFAS,  the  Army  combat 
developer  and  user  representative,  and  the  Marine  Corps  System  Command,  the 
Marine  Corps  materiel  developer,  have  assessed  the  AFATDS  Program  as 
essential  for  providing  integrated  fire  support  functionality  to  the  battlefield; 
however,  both  agree  that  the  AFATDS  Version  1  software  will  not  satisfy  their 
requirements.  The  AFATDS  Version  2  Program  has  not  had  a  dedicated  phase 
of  Engineering  and  Manufacturing  Development  to  achieve  production  hardware 
and  software  configurations  suitable  for  deployment.  The  USAFAS  stated  that 
the  Version  1  software  was  only  intended  to  be  fielded  to  the  test  unit  for  a  year 
and  then  replaced  with  Version  2.  Additionally,  USAFAS  stated  that  Version  1 
does  not  offer  sufficient  functionality  to  warrant  fielding  past  the  test  unit  and 
needs  an  additional  9  months  of  development  before  its  scheduled  IOT&E.  The 
Marine  Corps  will  not  field  the  AFATDS  Version  1  software  because  Version  1 
does  not  adequately  address  its  fire  support  needs.  The  Marine  Corps  has 
assessed  that  the  AFATDS  Program  needs  an  additional  year  of  development 
before  it  will  be  ready  for  IOT&E.  The  AMSAA  stated  that  the  AFATDS 
Program  faces  a  significant  risk  of  IOT&E  failure  if  the  development  schedule 
is  not  extended  by  at  least  6  months.  A  dedicated  phase  of  engineering  and 
manufacturing  development  should  address  the  lack  of  functional  maturity  in  the 
AFATDS  Program.  The  objective  should  be  to  develop  a  version  of  software 
capable  of  satisfying  user  requirements. 

Schedule  Breach.  The  AFATDS  Program  Office  has  not  reported  a  schedule 
breach  or  revised  its  acquisition  strategy.  The  DoD  Instruction  5000.2  states 
that  a  current  estimate  that  fails  to  meet  a  cost,  schedule,  or  performance 
threshold  constitutes  a  reportable  program  deviation..  Further,  the  Instruction 
states  that  the  program  manager  should  immediately  issue  a  program  deviation 
report  to  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology  when 
the  program  will  encounter  a  schedule  delay  of  180  days  or  more.  The 
AFATDS  Program  must  hold  its  Milestone  m,  Production  Approval,  decision 
no  later  than  December  31,  1994,  or  breach  its  baseline  schedule.  The 
AFATDS  Milestone  m  decision  was  originally  scheduled  for  June  1994  and 
rescheduled  for  December  1994;  however,  the  AFATDS  Program  mil  not  be 
mature  enough  for  a  Milestone  in  decision  in  December  1994,  due  in  part  to  a 
lack  of  functionality.  The  key  event  in  adhering  to  the  schedule  is  holding  the 
IOT&E  in  July  1994  to  prove  the  system  is  mature  enough  to  proceed  into 
production.  The  USAFAS,  the  Marine  Corps,  and  AMSAA  have  stated  that  the 
AFATDS  Program  needs  from  6  to  12  months  of  additional  development  before 
the  AFATDS  Program  will  be  ready  for  an  IOT&E.  Additionally,  due  to  the 
lack  of  functionality,  the  AFATDS  Program  cannot  pass  an  IOT&E  unless  the 
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test  is  designed  to  assess  what  the  system  can  do  rattier  than  what  it  should  do. 
However,  the  AFATDS  Program  Office: 

o  will  hold  the  IOT&E  on  a  release  of  the  Version  1  software  that  did 
not  go  through  developmental  testing, 

o  will  conduct  the  IOT&E  on  hardware  that  does  not  represent  the 
system  to  be  tested  due  to  the  lack  of  availability,  and 

o  will  conduct  the  IOT&E  on  a  software  design  that  does  not  meet  user 
requirements. 

Therefore,  the  scheduled  AFATDS  IOT&E  will  not  provide  a  valid  estimate  of 
the  operational  effectiveness  and  suitability  of  the  AFATDS  Program  because 
the  Version  1  software  lacks  the  functionality  to  replace  TACFIRE  or  satisfy 
user  requirements.  As  a  result,  a  deployable  version  of  AFATDS  software  will 
not  be  available  before  December  31,  1994. 

By  the  Army  not  reporting  the  breach,  the  Under  Secretary’s  ability  to  provide 
guidance  is  impaired.  Additionally,  the  software  design  will  not  satisfy  user 
requirements  and  the  conditions  that  caused  the  schedule  slippage  may  continue 
and  delay  the  fielding  of  a  software  version  capable  of  satisfying  user 
requirements.  Further,  the  $4.6  million  spent  on  the  IOT&E  would  be  wasted. 
The  AFATDS  Program  Office  should  submit  a  program  deviation  report  to  the 
Under  Secretary  of  Defense  for  Acquisition  and  Technology.  After  the  Army 
reports  the  breach,  the  Under  Secretary  should  convene  a  Defense  Acquisition 
Board13  review  of  the  AFATDS  Program  to  ascertain  the  cause  of  the  breach 
and  what  corrective  actions  must  be  taken. 

Development  Test  and  Evaluation  Programs.  The  AFATDS  developmental 
test  and  evaluation  program  is  based  on  satisfying  schedule  concerns  rather  than 
ensuring  that  the  AFATDS  Program  is  sufficiently  mature  to  enter  operational 
testing.  The  DoD  Instruction  5000.2  requires  that  a  program's  developmental 
test  and  evaluation  program  verify  that  the  system  is  ready  for  operational  test 
and  evaluation.  Key  developmental  tests  for  the  AFATDS  Program  are  the 
FDT&E  and  the  formal  qualification  test  (Appendix  D);  however,  the  tests  do 
not  prove  the  AFATDS  Program  is  ready  for  operational  test  and  evaluation. 


13The  DoD  Instruction  5000.2  states  that  the  Defense  Acquisition  Board  is  the  senior  advisory 
body  to  Under  Secretary  of  Defease  for  Acquisition  and  Technology  to  advise  the  Under 
Secretary  in  enforcing  policies  and  procedures  governing  the  operations  of  the  DoD  Acquisition 
System.  The  Defense  Acquisition  Board  is  the  primary  forum  to  advise  the  Under  Secretary  on 
mission  needs  approved  by  the  Joint  Requirements  Oversight  Council,  possible  Concept 
FYplnratinn  or  Definition  study  efforts,  and  Milestone  I  through  IV  decision  point  reviews  and 
program  reviews  of  major  Defense  acquisition  programs  subject  to  Defense  Acquisition  Board 
review.  The  reviews  ensure  that  a  program  is  ready  to  proceed  into  more  advanced  stages  of 
development  or  production  before  receiving  Milestone  approval  and  that  proposed  program 
plans  for  subsequent  stages  are  consistent  with  sound  acquisition  management  practices. 
Three  Defense  Acquisition  Board  committees  support  the  Defense  Acquisition  Board  review 
process. 
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Force  Development  Test  and  Experimentation.  The  FDT&E,  a 
developmental  test,  began  in  February  1994  with  contractor  personnel  who  are 
providing  the  user  with  essential  training  that  will  be  used  in  the  IOT&E.  hie 
AFATDS  Program  Office  will  allow  the  AFATDS  Program  to  transition  from 
software  development  to  FDT&E  without  assuring  that  key  events  were 
successfully  completed.  Specifically,  the  software  will  be  incomplete  and  toe 
target  computers  will  be  unavailable  for  toe  testing.  _  Magnavox  providedtoe 
AFATDS  Program  Office  with  functionality  matrices  on  Augusts,  1993, 
stating  that  toe  CONOPS  and  other  software  functionality  wouid  not  be 
available  for  testing  with  Version  1  software  at  toe  FDT&E.  CONOPS 
software  offers  new  and  essential  functionality  that  will  allow  AFATDS 
operational  facilities  to  automatically  take  over  processing  capabilities  for  each 
other  due  to  movement  or  catastrophic  loss.  Additionally,  toe  AFATDS 
software  will  not  have  been  written  for  toe  target  computers  in  time  for  toe 
FDT&E  The  AFATDS  Program  Office  does  not  have  enough  target  computers 
to  run  toe  FDT&E.  Specifically,  toe  FDT&E  will  be  conducted  usmg  toe 
382  TCUs  and  no  LCUs. 


The  FDT&E  will  not  satisfy  its  objective  of  certifying  toe  system  as  ready  to 
enter  toe  IOT&E.  Not  allowing  toe  AFATDS  software  and  hardware  to  mature 
before  toe  FDT&E  contributes  to  toe  risk  that  toe  AFATDS  Program  could  fad 
toe  IOT&E.  Specifically,  AMSAA  stated  that  toe  FDT&E  for  toe  Version  1 
software  has  a  high  risk  of  not  achieving  its  objective  of  certifying  that  toe 
AFATDS  Program  is  ready  for  IOT&E  because  toe  CONOPS  functionality  will 
not  be  tested  at  toe  FDT&E.  Further,  AMSAA  stated  that  toe  risk  of  fading  toe 
IOT&E  would  be  greatly  reduced  if  all  required  AFATDS  Version  1  software 
functionality  was  written  for  toe  target  computers  and  tested  at  an  FDT&E. 
AMSAA  believes  that  the  FDT&E  will  provide  good  user  feedback  to  enhance 
toe  engineering  and  manufacturing  development  of  toe  AFATDS  Program. 

Formal  Qualification  Test.  The  formal  qualification  test  will  not  be 
done  in  accordance  with  toe  AFATDS  Statement  of  Work,  which  requires  toe 
test  to  be  on  toe  target  computers  to  be  fielded.  The  formal  qualification  test 
will  be  done  on  toe  final  release  of  toe  software  to  toe  extent  it  is  differentfrom 
prior  releases  and  errors  are  known  to  exist;  however,  toe  AFATDS  Software 
Development  Plan  states  that  toe  formal  qualification  test  should  occur  on  toe 
entire  release  of  software  to  be  fielded.  A  full  formal  qualification  test  on  toe 
final  version  of  software  is  required  to  verify  that  no  catastrophic  errors  exist  in 
toe  software.  The  AFATDS  Program  Office  has  not  allowed  sufficient  time  for 
toe  formal  qualification  test  to  verify  that  all  catastrophic  errors  are  found  and 
corrected.  Therefore,  if  toe  software  is  not  tested  properly  during  engineering 
and  manufflf>-tnring  development,  the  software  could  experience  avoidable 
mission  failures. 


Initial  Operational  Test  and  Evaluation.  Functionality  called  for  m  toe 
AFATDS  ORD  will  not  be  available  in  toe  Version  1  software.  However,  in 
July  1994,  the  AFATDS  Program  Office  intends  to  begin  IOT&E  for  Version  1 
software  that  will  involve  from  1  to  2  months  of  testing.  The  IOT&E  is 
estimated  to  cost  $4.6  million.  The  DoD  Instruction  5000.2  states  that 
operational  test  and  evaluation  programs  shall  be  structured  to  determine  toe 
operational  effectiveness  and  suitability  of  a  system  under  realistic  combat 
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conditions  and  to  determine  whether  the  minimum  acceptable  operational 
performance  requirements  as  specified  in  the  operational  requirements  document 
have  been  satisfied.  Without  required  functionality,  the  Army  will  not  meet  the 
ORD  requirements  during  the  IOT&E. 

The  Version  1  software  will  not  offer  the  minimum  performance  requirements 
as  stated  in  the  ORD  such  as  close  air  support  and  naval  gunfire.  The  IOT&E 
will  be  conducted  on  computers  that  are  not  representative  of  the  AFATDS 
computers  to  be  fielded  because  the  AFATDS  Program  Office  did  not  allow 
enough  time  to  purchase  the  target  computers.  The  IOT&E  will  be  conducted 
using  58  computers  consisting  of  28  RISC  TCUs  (48  percent),  23  382  TCUs 
(40  percent),  and  the  7  LCUs  (12  percent).  The  28  RISC  TCUs  represent  all 
RISC  TCUs  available,  with  the  exception  of  2  spares.  The  lack  of  RISC  TCUs 
forces  the  AFATDS  Program  Office  to  use  the  382-TCU  for  40  percent  of  total 
computers  to  meet  the  IOT&E  date;  however,  the  382-TCUs  will  not  be  fielded. 
The  OSD  Director,  Operational  Test  and  Evaluation,  will  evaluate  the  IOT&E 
results.  Personnel  from  the  Office  of  the  Director,  Operational  Test  and 
Evaluation,  stated  that  the  degree  to  which  the  382-TCUs  are  used  could  nullify 
the  tests,  depending  on  the  extent  to  which  using  the  382-TCUs  may  obscure  the 
test  results.  Further,  AMSAA  has  rated  the  IOT&E  as  a  high-risk  event  due  to 
the  lack  of  necessary  FDT&E  testing  on  the  AFATDS  target  computer  and 
completed  Version  1  software. 

Documentation.  The  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications  and  Intelligence)  expressed  concern  that  the  managers  of  the 
Army  Tactical  Command  and  Control  Systems  need  to  keep  OSD 
decisionmakers  informed  on  the  Systems'  progress.  However,  the  AFATDS 
Program  Office  did  not  prepare  or  did  not  adequately  prepare  acquisition 
documentation  that  would  allow  decisionmakers  to  assess  the  AFATDS  Program 
properly.  The  computer  resources  life-cycle  management  plan  (CRLCMP) 
(Appendix  B),  COEA,  ORD,  and  TEMP  do  not  adequately  reflect  the  status  of 
the  AFATDS  Program. 

Computer  Resources  Life-Cycle  Management  Plan.  The  AFATDS 
Project  Office  did  not  complete  and  maintain  a  CRLCMP.  The  DoD 
regulations  require  the  preparation  of  a  CRLCMP  to  adequately  detail  the  major 
decisions  made  in  the  acquisition  of  computer  resources.  The  AFATDS 
Program  Office  made  performance  decisions  from  the  objective  system  to  arrive 
at  the  Version  1  software;  however,  the  AFATDS  Program  Office  did  not 
prepare  a  CRLCMP  to  document  the  performance  decisions  so  that 
decisionmakers  could  assess  the  basis  for  the  decisions.  Further,  the 
information  to  be  gathered  by  the  CRLCMP  was  not  available  to  prepare 
milestone  documents  such  as  the  COEA,  TEMP,  ORD,  integrated  program 
summary,  and  integrated  logistics  support  plan.  In  January  1994,  the  AFATDS 
Program  Office  started  preparing  a  CRLCMP. 

Cost  and  Operational  Effectiveness  Analysis.  The  Army  Training  and 
Doctrine  Command  began  updating  the  AFATDS  COEA  in  November  1993  and 
will  deliver  it  on  November  30,  1994,  providing  the  decisionmaker  with  the 
document  approximately  1  month  before  the  Milestone  III  decision  meeting. 
Continued  development  of  the  IFSAS  Program  is  not  detailed  in  the  COEA 
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alternatives.  The  IFSAS  Program  is  intended  to  provide  fire  support  automation 
pending  AFATDS'  delivering  desired  fire  support  functionality.  However,  all 
COEA  alternatives  require  the  IFSAS  Program  to  have  enhanced  capabilities  to 
satisfy  mission  requirements  because  AFATDS'  development  could  be  stopped 
at  Version  1  or  would  require  additional  years  to  develop  a  fieldable  system. 
The  AFATDS'  COEA  assumes  that  IFSAS  is  a  proven  system  that  can  satisfy 
requirements;  however,  it  does  not  alert  the  decisionmaker  that  additional 
development  is  necessary  to  convert  the  software  into  Ada14  programming 
language  and  will  require  additional  software  code  to  satisfy  mission 
requirements  such  as  coordinating  close  air  support  and  naval  gunfire. 
Therefore,  each  alternative  should  also  disclose  what  further  development  will 
be  required  of  IFSAS  so  that  decisionmakers  can  determine  whether  the  Army 
must  field  two  fire  support  systems.  If  the  alternatives  regarding  IFSAS  are  not 
disclosed  fully,  the  decisionmakers  may  not  consider  the  most  technologically  or 
economically  feasible  solution  to  fire  support  automation,  given  the  limited  time 
available  to  review  the  COEA. 

Operational  Requirements  Document.  The  ORD  specifies  what 
minimum  requirements  are  necessary  to  field  the  objective  version  of  the 
AFATDS  software.  However,  the  ORD  does  not  specify  the  requirements  for 
fielding  Version  1,  such  as: 

o  what  minimum  fire  support  functionality  or  what  minimum 
hardware  speed  and  memory  requirements  must  be  available  to  field  AFATDS, 

o  what  type  of  software  architecture  is  desirable,  and 

o  what  are  the  time-frame  availability  or  contingency  issues. 
For  example,  the  ORD  does  not  consider  the  availability  of  IFSAS  on  the 
requirements  for  AFATDS. 

The  present  ORD  does  not  provide  the  decisionmaker  with  a  comprehensive  set 
of  requirements  to  determine  whether  the  AFATDS  Version  1  software  is 
suitable  for  fielding  for  other  than  evaluation  and  feedback.  The  testers  cannot 
design  an  IOT&E  to  adequately  assess  the  AFATDS  Program  for  fielding 
because  the  artillery  target  intelligence,  naval  gunfire,  close  air  support, 
hardware  speed  ana  memory,  and  system  architecture  are  undefined  in  the 
ORD.  As  a  result,  the  IOT&E  is  only  a  test  of  what  functionality  AFATDS 
Version  1  software  has,  not  an  evaluation  of  ORD  requirements.  The  ORD 
should  be  updated  to  reflect  minimum  requirements  for  fielding  AFATDS. 

Test  and  Evaluation  Master  Plan.  The  AFATDS  TEMP  does  not 
identify  potential  operational  and  technical  limitations  of  the  alternative  concepts 
and  the  design  options  being  pursued  or  support  the  decision  to  certify  the 
AFATDS  Program  as  ready  for  operational  testing.  The  DoD  Instruction 
5000.2  states  that  minimum  test  planning  must  address  all  system  components 
that  are  critical  to  the  achievement  and  demonstration  of  contract  technical 
performance  specifications  and  minimum  acceptable  operational  performance 

l^Ada  is  the  only  programming  language  to  be  used  in  new  Defease  systems  and  major  software 
modifications  of  existing  systems  regardless  of  size,  cost,  or  functional  application. 
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requirements  specified  in  the  operational  requirements  document.  The  TEMP 
designs  a  testing  plan  for  the  Version  1  software  even  though  the  software  by 
design  cannot  meet  minimum  operational  requirements.  For  example,  the  ORD 
requires  that,  at  a  minimum,  the  AFATDS  Program  provide  close  air  support 
and  naval  gunfire  functionality.  The  AFATDS  Version  1  software  cannot 
adequately  meet  this  requirement,  yet  the  TEMP  is  an  attempt  to  design  a  test 
plan  that  would  result  in  fielding  the  AFATDS  Version  1  software.  The  TEMP 
does  not  test  the  ability  of  the  system  to  meet  ORD  requirements.  Rather,  the 
TEMP  focuses  on  the  functionality  designed  into  Version  1.  The  TEMP  should 
be  updated  to  verify  that  it  is  compatible  with  the  updated  ORD. 


Effect  of  Milestone  Decision 


Decisionmakers  will  not  have  all  information  to  make  fully  informed  decisions, 
which  could  result  in  cost  and  performance  problems. 

Decision  Authority.  After  completing  the  Milestone  III  review,  the  Army 
plans  to  transfer  decision  authority  for  subsequent  reviews  to  the  Program 
Executive  Officer,  Command  and  Control  Systems,  before  the  AFATDS 
Program  is  mature.  The  DoD  Instruction  5000.2  states  that  the  lowest  level 
decision  authority  for  an  Army  ACATIC  program  shall  be  the  Army 
Acquisition  Executive.  However,  in  view  of  the  AFATDS  Program 
deficiencies,  the  Milestone  in  decision  should  be  made  by  the  Under  Secretary 
of  Defense  for  Acquisition  and  Technology. 

Acquisition  Strategy.  The  Army's  acquisition  decisions  are  being  made  to 
adhere  to  schedule  considerations,  which  increase  the  risk  that  the  AFATDS 
Program  will  not  satisfy  user  needs.  The  acquisition  strategy  will  not  allow  the 
user  sufficient  time  to  evaluate  the  system  before  final  decisions  are  made. 
Further,  the  user  will  not  be  able  to  add  functionality  once  the  AFATDS 
Program  is  fielded  because  of  computer  speed  and  memory  constraints.  As  a 
result,  tiie  fielded  AFATDS  Program  will  not  provide  the  needed  functionality, 
causing  the  Government  significant  cost,  schedule,  and  performance  problems. 
Specifically,  the  Government  could  prematurely  procure  hardware  that  does  not 
meet  requirements.  Deferring  computer  procurement  until  definite  decisions  on 
fielding  the  AFATDS  Program,  including  which  version,  will  permit 
$72.3  million  (Appendix  E)  to  be  put  to  better  use  over  the  succeeding  6-year 
period. 

Initial  Fire  Support  Automated  System.  The  Army's  failure  to  adequately 
disclose  that  IFSAS  is  a  recently  fielded  alternative  to  TACFIRE,  pending 
AFATDS  showing  the  desired  functionality,  could  cause  decisionmakers  to 
make  less  than  fully  informed  decisions.  The  decision  to  rush  AFATDS  into 
the  field  would  cause  the  Army  to  support  two  new  systems  for  the  same 
mission. 

Test  and  Evaluation.  The  Army's  failure  to  adequately  perform  developmental 
tests  on  all  portions  of  the  AFATDS  Version  1  software  would  lead  to  the 
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decision  to  send  the  software  to  operational  testing  prematurely.  As  a  result, 
technical  errors  overlooked  at  developmental  testing  may  not  be  discovered  until 
the  system  is  fielded  because  operational  testing  is  not  designed  for  this  activity. 
Errors  discovered  at  operational  testing  could  cause  the  system  to  fail  the 
IOT&E,  which  would  inefficiently  use  the  $4.6  million  dedicated  for  the  test, 
require  an  unknown  amount  of  development  funds  to  correct  errors,  and  delay 
schedule.  The  Army  not  conducting  the  IOT&E  with  production-representative 
computers  may  cause  the  decisionmakers  to  pass  or  fail  the  system  incorrectly, 
which  would  also  impact  cost,  schedule,  and  performance. 

Capabilities.  The  Army  not  adequately  addressing  the  minimum  capabilities 
necessary  to  field  the  AFATDS  Version  1  software  in  its  acquisition 
documentation  hinders  the  decision  authority  in  making  a  fully  informed 
decision.  The  fielding  of  the  Version  1  software  would  result  in  a  lack  of 
functionality  in  regard  to  artillery  target  intelligence,  close  air  support,  and 
naval  gunfire.  Additionally,  the  inadequate  memory  and  system  architecture 
would  hamper  system  performance,  such  as  communications  through  the 
CONOPS  software.  As  a  result,  the  Army  would  have  to  spend  more  funds  to 
correct  those  problems  than  if  the  requirements  were  adequately  defined  and 
performed  as  early  as  possible. 


Conclusion 


The  lack  of  functionality  necessitates  cancellation  of  the  December  1994 
planned  Milestone  in,  Production  Approval,  decision  and  the  planned 
procurement  of  computers  until  the  AFATDS  software  has  reached  a  sufficient 
level  of  maturity.  The  schedule  baseline  breach  should  be  reported  in  the 
program  deviation  report,  Defense  Acquisition  Executive  Summary,  and  the 
Selected  Acquisition  Reports  because  the  software  has  not  reached  a  level  of 
maturity  suitable  for  fielding  and,  therefore,  has  a  reportable  breach.  After 
reporting  the  breach,  a  Defense  Acquisition  Board  program  review  should  be 
held  to  assess  alternatives  for  meeting  user  requirements,  which  would  include 
subsequent  versions  of  the  AFATDS,  and  approve  a  restructured  acquisition 
strategy  with  a  dedicated  engineering  and  manufacturing  development  phase  for 
any  developmental  solution.  Further,  the  AFATDS  ORD  should  be  updated  to 
address  minimum  acceptable  operational  requirements,  interoperability  with  the 
IFSAS  Program,  and  Marine  Corps  requirements.  The  TEMP  should  be 
revised  to  reflect  the  minimum  operating  requirements  as  reflected  in  the  ORD. 
Also,  the  COEA  should  contain  alternatives  that  reflect  the  impact  of  IFSAS  on 
the  AFATDS  Program  and  be  completed  before  the  recommended  Defense 
Acquisition  Board  program  review. 
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Recommendations  for  Corrective  Action 

We  recommend  that  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology: 

1.  Redesignate  the  Advanced  Field  Artillery  Tactical  Data  System  as  an 
acquisition  category  ID  program. 

2.  Cancel  the  Army  System  Acquisition  Review  Council  Milestone  in, 
Production  Approval,  decision  planned  for  Version  1  of  the  Advanced  Field 
Artillery  Tactical  Data  System. 

3.  Cancel  procurement  plans  for  hardware  to  support  Version  1  of  the 
Advanced  Field  Artillery  Tactical  Data  System. 

4.  Direct  the  Army  to  report  the  schedule  baseline  breach. 

5.  Conduct  a  Defense  Acquisition  Board  program  review  of  alternatives  for 
mooting  user  requirements,  including  subsequent  versions  of  the  Advanced 
Field  Artillery  Tactical  Data  System,  and  approval  of  a  restructured 
acquisition  strategy  including  a  dedicated  engineering  and  manufacturing 
development  phase  for  any  developmental  solution. 

6.  Require  the  Advanced  Field  Artillery  Tactical  Data  Systran  Operational 
Requirements  Document  be  updated  to  address  minimum  acceptable 
operational  requirements,  including  interoperability  with  the  Initial  Fire 
Support  Automation  System  and  with  the  Navy,  the  Air  Force,  and  the 
Marine  Corps. 

7.  Revise  the  Test  and  Evaluation  Master  Plan  to  reflect  the  minimum 
operating  requirements  in  the  updated  Operational  Requirements 
Documents. 

8.  Complete  the  Cost  and  Operational  Effectiveness  Analysis  before  the 
recommended  Defense  Acquisition  Board  program  review. 


Management  Comments  and  Audit  Response 


Management  Comments.  We  did  not  receive  comments  from  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology  to  a  draft  of  this  report 
issued  March  17,  1994.  The  comments  were  required  by  May  16,  1994. 

Audit  Response.  The  DoD  Directive  7650.3  requires  that  all  audit 
recommendations  be  resolved  promptly.  Therefore,  we  request  that  the  Under 
Secretary  provide  comments  on  the  final  report. 
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Part  III  -  Additional  Information 


Appendix  A.  Milestone  HI,  Production 

Approval,  Decision 

The  DoD  Instruction  5000.2,  "Defense  Acquisition  Policies  and  Procedures," 
February  23,  1991,  states  that  a  program  may  not  enter  full-rate  production  and 
deployment  unless  the  milestone  decision  authority  confirms  that: 

o  the  system  threat  assessment  and  performance  objectives  and 
thresholds  have  been  validated; 

o  test  results  and  low-rate  initial  production  provide  reasonable 
assurance  that  the  design  is  stable,  operationally  acceptable,  logistically 
supportable,  and  capable  of  being  produced  efficiently; 

o  potential  environmental  consequences  of  the  program  have  been 
analyzed  and  appropriate  mitigating  measures  have  been  developed; 

o  projected  life-cycle  costs  and  annual  funding  requirements  are 
affordable  in  the  context  of  long-range  investment  or  similar  plans;  and 

o  adequate  personnel  and  funds  have  been  programmed  to  support 
production,  deployment,  and  support. 

At  Milestone  HI,  the  milestone  decision  authority  decides  on  entry  into  the 
production  and  deployment  phase  of  the  acquisition  process.  The  objectives  of 
this  phase  are  to: 

o  establish  a  stable,  efficient  production  and  support  base; 

o  achieve  an  operational  capability  that  satisfies  the  mission  need;  and 

o  conduct  follow-on  operational  and  production  verification  testing  to 
confirm  and  monitor  performance  and  quality  and  verify  the  correction  of 
deficiencies. 
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Computer  Resources  Life-Cycle  Management  Plan.  The  DoD  Instruction 
5000.2,  "Defense  Acquisition  Policies  and  Procedures,"  February  23,  1991, 
states  that  a  computer  resources  life-cycle  management  plan  (CRLCMP) 
documents  the  management  approach,  decisions,  and  plans  associated  with 
computer  resources.  Further,  the  CRLCMP  identifies  mid  addresses  critical 
issues,  objectives,  risks,  methodologies,  and  evaluation  criteria.  The  CRLCMP 
also  structures  development,  test,  quality  assurance,  and  support  processes  to 
provide  data  that  permit  quantitative  assessment  of  the  impact  of  computer 
resources  on  weapon  system  cost,  schedule,  and  performance. 

The  Defense  Systems  Management  College  defines  the  CRLCMP  in  its  Mission- 
Critical  Computer  Resources  Management  Guide  (MCCRMG)  as  one  of  the 
most  important  documents  to  reduce  software  life-cycle  costs.  It  further  states 
that  the  CRLCMP: 

o  is  to  be  developed  early  in  the  acquisition  cycle  to  ensure  that  all 
issues  and  resources  relevant  to  weapon  system  acquisition,  testing,  and  support 
are  accounted  for  properly; 

o  defines  the  criteria  for  measuring  progress  and  identifies  the  resources 
needed  to  develop,  test,  acquire,  and  support  computer  resources,  such  as 
facilities,  personnel,  hardware,  software,  training,  funding,  and  tools;  and 

o  should  be  updated  whenever  the  software  is  modified  or  at  least 
annually. 

Cost  of  Software  Fixes.  The  MCCRMG  stated  that  the  costs  of  fixing  software 
errors  early  in  the  development  of  the  software  is  insignificant  compared  to  the 
costs  of  finding  and  correcting  the  error  once  the  software  has  been  delivered. 
Specifically,  the  MCCRMG  estimated  that  the  costs  to  correct  software 
problems  increases  as  a  system  proceeds  from  requirements  analysis  through 
design  and  test  to  deployment. 

Hardware  Decisions.  The  DoD  Instruction  5000.2  states  that  a  program  office 
will  not  finalize  computer  hardware  resource  decisions  until  the  software  design 
is  mature  enough  to  minimize  the  risk  that  the  computer  selected  has  inadequate 
speed  and  memory  capacity,  which  is  usually  measured  by  the  amount  of 
random  access  memory.  Additionally,  software  design  and  hardware  design 
schedules  must  be  closely  linked.  The  MCCRMG  stated  that  the  hardware  at 
delivery  should  have  at  least  a  50  percent  speed  and  memory  reserve. 

Memory.  Memory  is  an  electronic  circuit  that  allows  for  the  storage  and 
retrieval  of  information.  Memory  can  also  refer  to  external  data  storage 
systems  such  as  disk  drives  or  tape  drives;  however,  "memory"  usually  refers  to 
the  storage  of  random  access  memory  that  is  directly  connected  to  the 
computer’s  processor.  Random  access  memory  is  semiconductor-based  memory 
that  can  be  read  and  written  to  by  the  computer  or  other  access  devices.  The 
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data  in  random  access  memory  is  lost  when  the  power  is  disconnected  from  the 
system.  Usually  the  software  program  that  operates  the  computer  is  stored  m 
random  access  memory  to  increase  speed  in  operation. 

Software  Development.  At  the  beginning  of  the  software  development 
process,  the  software  developer  does  not  use  as  much  code  and  associated 
paperwork.  However,  as  the  software  begins  computer-based  testing,  major 
resources  such  as  computers,  weapon  system  hardware,  technicians,  and 
systems  analysts  are  involved  and  costs  can  escalate  quickly. 

Software  Impact.  The  MCCRMG  stated  that,  as  of  1980,  software  is 
50  percent  of  system  design  effort;  therefore,  software  is  no  longer  just  a  part  of 
a  system  but  is  a  system  in  its  own  right,  having  the  integration  function  for 
various  subsystems  of  a  weapon.  During  the  past  30  years,  the  computer 
hardware-to-software  ratio  as  a  percentage  of  computer  resource  cost  has 
reversed  from  80  percent  to  20  percent  hardware-to-software  to  20  percent  to 
80  percent  hardware  to  software.  The  software  cost  estimate  also  reflects 
software  support,  which  accounts  for  as  much  as  60  percent  of  the  cost.  The 
MCCRMG  estimated  that  by  1995  DoD  software  costs  will  approach 
$35.7  billion,  up  from  $11.4  billion  in  1985.  The  rise  in  the  cost  of  software 
can  be  attributed  to  the  fact  that  software  design  and  support  are  labor  intensive. 
Although  software  allows  automation  of  many  tasks,  very  few  machines  can 
generate  computer  programs  from  a  set  of  requirements;  therefore,  software 
development  and  support  require  programmers  and  other  technical  specialists. 
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Appendix  C.  Acquisition  Community 

Members  of  the  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS) 
acquisition  community  include: 

o  the  Program  Executive  Officer,  Command  and  Control  Systems,  who 
is  responsible  for  all  Army  Tactical  Command  and  Control  System  activities  and 
reports  directly  to  the  Army  Acquisition  Executive; 

o  the  Project  Manager,  Field  Artillery  Tactical  Data  Systems,  who  is 
responsible  for  the  management  of  the  AFATDS  Program; 

o  the  U.S.  Army  Field  Artillery  School  (USAFAS),  the  combat 
developer,  acts  as  the  user  representative  for  AFATDS; 

o  the  Marine  Corps  Systems  Command,  the  materiel  developer  for 
Marine  Corps  Fire  Support  Command  and  Controls  Systems,  is  responsible  for 
the  Marine  Corps’  participation  in  the  AFATDS  Program; 

o  the  Army  Materiel  Systems  Analysis  Activity  (AMSAA),  the 
AFATDS  independent  technical  evaluator;  and 

o  the  U.S.  Army  Operational  Evaluation  Command,  the  independent 
operational  evaluator. 
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Appendix  D.  Test  and  Evaluation 

Developmental  Test  and  Evaluation.  The  DoD  Instruction  5000.2,  "Defense 
Acquisition  Policies  and  Procedures,"  February  23,  1991,  states  that 

developmental  test  and  evaluation  programs  shall: 

o  identify  potential  operational  and  technological  limitations  of  the 
alternative  concepts  and  design  options  being  pursued, 

o  support  the  identification  of  cost  performance  trade-offs, 

o  support  the  identification  and  descriptions  of  design  risks, 

o  substantiate  that  contract  technical  performance  and  manufacturing 
process  requirements  have  been  achieved,  and 

o  support  the  decision  to  certify  the  system  is  ready  for  operational  test 
and  evaluation. 

Developmental  test  and  evaluation  of  software  can  be  accomplished  by  force 
development  testing  and  experimentation  and  formal  qualification  testing. 

Force  Development  Testing  and  Experimentation.  For  the  AFATDS 
Program,  the  force  development  testing  and  experimentation  is  a  training 
exercise  to  prove  the  system  is  ready  for  an  initial  operational  test  and 
evaluation.  The  user,  user  representatives,  contractor,  and  other  interested 
parties  participate  in  the  exercise. 

Formal  Qualification  Testing.  A  formal  qualification  test  is  a 
contractor-conducted  developmental  test,  witnessed  by  Government 
representatives,  intended  to  show  that  the  software  meets  the  contract 
specifications.  The  Defense  Systems  Management  College  personnel  stated  that 
a  full  formal  qualification  test  should  be  run  on  software  to  detect  possible 
catastrophic  errors  in  the  software  created  by  combining  computer  software 
configuration  items. 

Initial  Operational  Test  and  Evaluation.  The  DoD  Instruction  5000.2  states 
that  an  Initial  Operational  Test  and  Evaluation  consists  of  all  operational  tests 
and  evaluations  conducted  on  production  or  production-representative  articles  to 
support  a  decision  to  proceed  beyond  low-rate  initial  production.  An  Initial 
Operational  Test  and  Evaluation  is  conducted  to  provide  a  valid  estimate  of 
expected  system  operational  effectiveness  and  operational  suitability. 

Operational  Test  and  Evaluation.  The  DoD  Instruction  5000.2  states  that 
operational  test  and  evaluation  programs  shall  determine  the  suitability  of  a 
system  under  realistic  combat  conditions  and  whether  the  minimum  acceptable 
operational  performance  requirements  as  specified  in  the  operational 
requirements  document  have  been  satisfied.  Specifically: 

o  threat-representative  forces  shall  be  used  when  possible; 
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o  typical  users  shall  operate  and  maintain  the  system,  simulating  combat 
stress  and  peacetime  conditions;  and 

o  production  or  production-representative  articles  shall  be  used  for  the 
dedicated  phase  of  operation  test  and  evaluation  that  supports  the  full-rate 
production  decision. 
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Appendix  E.  Summary  of  Potential  Benefits 

Resulting  From  Audit 


Recommendation 

Reference  Description  of  Benefit 


Amount  and/or 
Type  of  Benefit 


1. 

Program  Results.  Will  provide  a 
sufficient  level  of  OSD  oversight  to 
ensure  requirements  are  achieved. 

Nonmonetary. 

2. 

Program  Results.  Will  delay  the 
Milestone  m,  Production  Approval, 
decision  and  related  Initial 

Operational  Test  and  Evaluation 
until  the  Program  is  mature. 

$4.6  million1  of  Initial 
Operational  Test  and 
Evaluation  funds  will 
be  put  to  better  use 
from  FYs  1994 
through  1999. 

3. 

Program  Results.  Will  delay  the 
procurement  of  hardware  until  the 
software  is  complete. 

$72.3  million2  in 
hardware  procurement 
funds  will  be  put  to 
better  use  from 

FYs  1994  through 
1999. 

4. 

Program  Results.  Will  report  the 
baseline  breach  and  allow  OSD 
decisionmakers  the  opportunity  to 
provide  direction  to  the  Program. 

Nonmonetary. 

5. 

Program  Results.  Will  determine 
Program  direction. 

Nonmonetary. 

6. 

Program  Results.  Will  update  the 
Operational  Requirements 

Document  to  adequately  reflect 
Program  requirements. 

Nonmonetary. 

1  Funds  put  to  better  use  by  fiscal  year  (Research,  Development,  Test  and  Evaluation,  $  in 
millions): 


1994  1995 

$4.6  $0.0 


1996  1997 

$0.0  $0.0 


1998  1999 

$0.0  $0.0 


Total 

$4.6 


2Funds  put  to  better  use  by  fiscal  year  (Other  Procurement,  Army,  $  in  millions): 

1994  1995  1996  1997  1998  1999  Total 

$7.2  $21.5  $18.9  $24.7  $0.0  $0.0  $72.3 
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Recommendation 

Reference  Description  of  Benefit 


Amount  and/or 
Type  of  Benefit 


7.  Program  Results.  Will  update  the  Nonmonetary. 

Test  and  Evaluation  Master  Plan  to 

adequately  reflect  Program 
requirements. 

8.  Program  Results.  Will  update  the  Nonmonetary. 

Cost  and  Operational  Effectiveness 

Analysis  to  adequately  reflect 
Program  alternatives. 


31 


Appendix  F.  Organizations  Visited  or  Contacted 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Washington,  DC 
Director,  Acquisition  Program  Integration,  Washington,  DC 
Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence), 
Washington,  DC 

Director,  Operational  Test  and  Evaluation,  Washington,  DC 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Research,  Development  and  Acquisition), 
Washington,  DC 

Program  Executive  Office,  Command  and  Control  Systems,  Fort  Monmouth,  NJ 
Project  Manager,  Field  Artillery  Tactical  Data  Systems,  Fort  Monmouth,  NJ 
Project  Manager,  Common  Hardware  and  Software,  Fort  Monmouth,  NJ 
Director  of  Information  Systems  for  Command,  Control,  Communications  and 
Computers,  Washington  DC 

Director,  Modernization  and  Integration,  Washington,  DC 
Army  Materiel  Command,  Washington,  DC 

Army  Materiel  Systems  Analysis  Activity,  Aberdeen  Proving  Ground,  MD 
Army  Operational  Test  and  Evaluation  Command,  Alexandria,  VA 
Army  Training  and  Doctrine  Command,  Fort  Monroe,  VA 

U.S.  Army  Training  and  Doctrine  Command,  Analysis  Command,  White  Sands 
Missile  Range,  NM 

U.S.  Army  Field  Artillery  School,  Fort  Sill,  OK 
Director  of  Combat  Development,  Fort  Sill,  OK 


Department  of  the  Navy 

Commandant  of  the  Marine  Corps,  Washington,  DC 
Marine  Corps  Systems  Command,  Quantico,  VA 

Program  Manager,  Command  and  Control,  Quantico,  VA 
Marine  Corps  Combat  Development  Command,  Quantico,  VA 
Director  of  Requirements  Division,  Quantico,  VA 
Space  and  Naval  Warfare  Systems  Command,  Arlington,  VA 
Naval  Sea  Systems  Command,  Arlington,  VA 

Deputy  Commander  for  Ship  Design  and  Engineering,  Arlington,  VA 
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Defense  Agencies 

Defense  Logistics  Agency,  Alexandria,  VA 

Defense  Contract  Management  Command,  Alexandria,  VA 

Defense  Plant  Representative  Office,  Magnayox,  Fort  Wayne,  IN 
Defense  Systems  Management  College,  Fort  Belvoir,  VA 


Contractor 

Magnavox  Electronic  Systems  Company,  Fort  Wayne,  IN 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 

Principal  Deputy  Under  Secretary  of  Defense  (Acquisition  and  Technology) 
Director,  Acquisition  Program  Integration 
Director,  Tactical  Systems 
Assistant  Secretary  of  Defense  (Economic  Security) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence) 
Comptroller  of  the  Department  of  Defense 
Director,  Operational  Test  and  Evaluation 
Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 

Director,  Program  Analysis  and  Evaluation 


Department  of  the  Army 

Secretary  of  the  Army  ... 

Assistant  Secretary  of  the  Army  (Research,  Development  and  Acquisition) 
Program  Executive  Office,  Command  and  Control  Systems 
Project  Manager,  Field  Artillery  Tactical  Data  System 
Army  Materiel  Command 

Army  Materiel  Systems  Analysis  Activity 
Army  Operational  Test  and  Evaluation  Command 
Army  Training  and  Doctrine  Command 

U.S.  Army  Training  and  Doctrine  Command,  Analysis  Command 
U.S.  Army  Field  Artillery  School 
Director  of  Combat  Development 
Auditor  General,  Department  of  die  Army 


Department  of  the  Navy 

Secretary  of  the  Navy 
Commandant  of  the  Marine  Corps 
Marine  Corps  Systems  Command 

Program  Manager,  Command  and  Control 
Marine  Corps  Combat  Development  Command 
Director  of  Requirements  Division 
Assistant  Secretary  of  the  Navy  (Financial  Management) 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 

Comptroller  of  the  Navy 

Auditor  General,  Naval  Audit  Service 
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Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Acquisition) 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Air  Force  Audit  Agency 


Defense  Agencies 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 

Commander,  Defense  Contract  Management  Command 

Defense  Plant  Representative  Office,  Magnavox,  Fort  Wayne,  IN 
Inspector  General,  Defense  Intelligence  Agency 
Inspector  General,  National  Security  Agency 
Commandant,  Defense  Systems  Management  College 
Director,  Defense  Logistics  Studies  Information  Exchange 


Non-Defense  Organizations 

Office  of  Management  and  Budget  .  a  ^ .  .  . 

U.S.  General  Accounting  Office,  National  Security  and  International  Affairs  Division, 
Technical  Information  Center 

Chairman  and  Ranking  Minority  Member  of  the  Following  Congressional  Committees 
and  Subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 

House  Subcommittee  on  Legislation  and  National  Security,  Committee  on 
Government  Operations 
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Audit  Team  Members 


Donald  E.  Reed 

Russell  A.  Rau 

Jack  D.  Snider 
Eric  L.  Lewis 
Cris  M.  Helfrich 
Scott  A.  Marx 
Mary  Ann  Hourcl6 
Teresa  D.  Bone 


Director,  Acquisition  Management 
Directorate 

Audit  Program  Director,  Systems 
Acquisition  Division 
Audit  Project  Manager 
Senior  Auditor 
Auditor 
Auditor 
Editor 

Administrative  Support 


